Logsearch with Kibana Me Logs

Introduction

Draining logs from your Cloud Foundry hosted application to a backend Logstash/Elastic Search is great - it drains them all day long. Except, you can't see them. They are stored in Elastic Search and you have no way to access either Elastic Search nor a Kibana UI to view the logs. The application hosts Kibana 3 and a proxy that binds to your Logstash/Elastic Search backend service. You can now see your logs in Kibana! It assumes that users are getting Logstash via the Docker/Logstash Service Broker.

Assumptions

It is assumed that:

you already have an application running on your own Cloud Foundry you are using the Docker/Logstash Service Broker (see below) or a service broker that provides matching credentials schema you have already bound your application to your logstash service instance and Cloud Foundry is already draining logs into it you have the Go programming language installed on your machine For example:

cf create-service logstash14 free my-logstash-service cf bind-service my-app my-logstash-service cf restart my-app You can confirm that your application is bound to the service:

$ cf services Getting services in org system / space dev as admin... OK

name service plan bound apps status my-logstash-service logstash14 free my-app available Usage

To view your application's logs in Kibana you need to deploy the kibana-me-logs application and also bind it to the same my-logstash-service service instance as above:

git clone https://github.sw.ge.com/cf-platform-services/kibana-me-logs.git cd kibana-me-logs cf push kibana-myapp --no-start --random-route -b https://github.com/heroku/heroku-buildpack-go.git cf bs kibana-myapp my-logstash-service cf start kibana-myapp Now view your Kibana UI in your browser. It should redirect to a url like http://kibana-myapp.apps.1.2.3.4.xip.io/#/dashboard/file/logstash.json automatically and start showing your logs.

Note - go build pack used - https://github.com/cloudfoundry/go-buildpack

If you are a regular Go user, you can also fetch the application using:

go get github.com/cloudfoundry-community/kibana-me-logs cd $GOPATH/src/github.com/cloudfoundry-community/kibana-me-logs